As an autonomous system performs a task, it should maintain a calibrated estimate of the probability that it will achieve the user's goal. If that probability falls below some desired level, it should alert the user so that appropriate interventions can be made. This paper considers settings where the user's goal is specified as a target interval for a real-valued performance summary, such as the cumulative reward, measured at a fixed horizon $H$. At each time $t \in \{0, \ldots, H-1\}$, our method produces a calibrated estimate of the probability that the final cumulative reward will fall within a user-specified target interval $[y^-,y^+].$ Using this estimate, the autonomous system can raise an alarm if the probability drops below a specified threshold. We compute the probability estimates by inverting conformal prediction. Our starting point is the Conformalized Quantile Regression (CQR) method of Romano et al., which applies split-conformal prediction to the results of quantile regression. CQR is not invertible, but by using the conditional cumulative distribution function (CDF) as the non-conformity measure, we show how to obtain an invertible modification that we call \textbf{P}robability-space \textbf{C}onformalized \textbf{Q}uantile \textbf{R}egression (PCQR). Like CQR, PCQR produces well-calibrated conditional prediction intervals with finite-sample marginal guarantees. By inverting PCQR, we obtain marginal guarantees for the probability that the cumulative reward of an autonomous system will fall within an arbitrary user-specified target intervals. Experiments on two domains confirm that these probabilities are well-calibrated.

One approach to meet the challenges of deep lifelong reinforcement learning (LRL) is careful management of the agent's learning experiences, to learn (without forgetting) and build internal meta-models (of the tasks, environments, agents, and world). Generative replay (GR) is a biologically inspired replay mechanism that augments learning experiences with self-labelled examples drawn from an internal generative model that is updated over time. We present a version of GR for LRL that satisfies two desiderata: (a) Introspective density modelling of the latent representations of policies learned using deep RL, and (b) Model-free end-to-end learning. In this paper, we study three deep learning architectures for model-free GR, starting from a na\"ive GR and adding ingredients to achieve (a) and (b). We evaluate our proposed algorithms on three different scenarios comprising tasks from the Starcraft-2 and Minigrid domains. We report several key findings showing the impact of the design choices on quantitative metrics that include transfer learning, generalization to unseen tasks, fast adaptation after task change, performance wrt task expert, and catastrophic forgetting. We observe that our GR prevents drift in the features-to-action mapping from the latent vector space of a deep RL agent. We also show improvements in established lifelong learning metrics. We find that a small random replay buffer significantly increases the stability of training. Overall, we find that "hidden replay" (a well-known architecture for class-incremental classification) is the most promising approach that pushes the state-of-the-art in GR for LRL and observe that the architecture of the sleep model might be more important for improving performance than the types of replay used. Our experiments required only 6% of training samples to achieve 80-90% of expert performance in most Starcraft-2 scenarios.

As the need for more sophisticated artificial intelligences (AIs) grows, the challenges that they must face along the way have to evolve accordingly. Real-time strategy (RTS) video games, unlike turn-based board games such as chess, can serve as a vast playground for pushing the limits of AI. In particular, StarCraft II (SC2), one of the world's most popular and skill-demanding RTS games, has already been the object of a few groundbreaking AI-related studies. In SC2 matches, each player has to build up and command an army of varied units to defeat their opponent using wit and grit. While AI-based systems can excel at many aspects of the game, improving their decision-making regarding when their units should be sent to or relocated during a battle is remarkably difficult.

Machine learning models have already mastered Chess, Go, Atari games and more, but in order for it to ascend to the next level, researchers at Facebook intend for AI to take on a different kind of game: the notoriously difficult and infinitely complex NetHack. "We wanted to construct what we think is the most accessible'grand challenge' with this game. It won't solve AI, but it will unlock pathways towards better AI," said Facebook AI Research's Edward Grefenstette. "Games are a good domain to find our assumptions about what makes machines intelligent and break them." You may not be familiar with NetHack, but it's one of the most influential games of all time.

Welcome to AI book reviews, a series of posts that explore the latest literature on artificial intelligence. For decades, we've been trying to develop artificial intelligence in our own image. And at every step of the way, we've managed to create machines that can perform marvelous feats and at the same time make surprisingly dumb mistakes. After six decades of research and development, aligning AI systems with our goals, intents, and values continues to remain an elusive objective. Every major field of AI seems to solve part of the problem of replicating human intelligence while leaving out holes in critical areas.

Over the past few years, data science has become popularized and more and more ingrained in every aspect of our lives. From mass marketing campaigns on Amazon to the recommendations tab on YouTube, data science has become the tool to track and analyze daily decisions. Even if you don't realize it at first, data science dictates a lot of different aspects of our lives. This versatility has led it to become one of the most sought out job positions and college majors. The ability to analyze data has become an important aspect of any job.

This is less of an AI question and more of a question about game theory. First, some terminology: In the world of Go, we have the concepts of the'Go God' and the'Go Devil'. The Go God always plays perfectly in response to the board state, but doesn't assume anything about its opponent. The Go Devil, on the other hand, has full knowledge of the sort of player it is playing against, and plays optimally to defeat that player. Clearly the Go Devil is stronger than the Go God, because when playing against the Go God (or against another Go Devil) he plays as if he is another Go God, and against all imperfect players he plays at least as well as the Go God and in some cases better.

DeepMind says this latest iteration of AlphaStar -- AlphaStar Final -- can play a full StarCraft 2 match under "professionally approved" conditions, importantly with limits on the frequency of its actions and by viewing the world through a game camera. It plays on the official StarCraft 2 Battle.net "StarCraft has been a grand challenge for AI researchers for over 15 years, so it's hugely exciting to see this work recognized in Nature," said DeepMind cofounder and CEO Demis Hassabis. "These impressive results mark an important step forward in our mission to create intelligent systems that will accelerate scientific discovery." DeepMind's forays into competitive StarCraft play can be traced back to 2017, when the company worked with Blizzard to release an open source tool set containing anonymized match replays.

DeepMind says it has created the first artificial intelligence to reach the top league of one of the most popular esport video games. It says Starcraft 2 had posed a tougher AI challenge than chess and other board games, in part because opponents' pieces were often hidden from view. Publication in the peer-reviewed journal Nature allows the London-based lab to claim a new milestone. But some pro-gamers have mixed feelings about it claiming Grandmaster status. DeepMind - which is owned by Google's parent company Alphabet - said the development of AlphaStar would help it develop other AI tools which should ultimately benefit humanity.

DeepMind today announced a new milestone for its artificial intelligence agents trained to play the Blizzard Entertainment game StarCraft II. The Google-owned AI lab's more sophisticated software, still called AlphaStar, is now grandmaster level in the real-time strategy game, capable of besting 99.8 percent of all human players in competition. The findings are to be published in a research paper in the scientific journal Nature. Not only that, but DeepMind says it also evened the playing field when testing the new and improved AlphaStar against human opponents who opted into online competitions this past summer. For one, it trained AlphaStar to use all three of the game's playable races, adding to the complexity of the game at the upper echelons of pro play.